#Two-Phase Commit (2PC)

[some-programming-pearls]

Explain what a two-phase commit is in terms of Microservice architectures?

Two-Phase Commit (2PC) is a distributed transaction protocol used to ensure atomicity and consistency across multiple microservices in a microservices architecture. In this context, 2PC helps maintain data integrity when a transaction involves interactions with multiple microservices, ensuring that either all microservices successfully complete their operations or none of them do. Here’s how…

[123widpress]

Transaction Security and Management of Blockchain-Based Smart Contracts in E-Banking-Employing Micro-segmentation and Yellow Saddle Goatfish

Authored by:- Wid Alaa Jebbar and Mishall AL-Zubaidie

for full-length paper press here.

Abstract:-

Our research attempts to improve the system in which banks deal with the security of financial transactions. This research leverages the idea of micro-segmenting the entire system into designated zones to concentrate on security, where each zone has its own rules and limitations. These rules are managed by a smart contract, which decides whether they have been observed to verify the legitimacy of the customer. First, the two-phase commit algorithm (2PC) was used to specify the type of e-banking request. After this, the micro-segmentation principle was applied to isolate each type of e-transaction process alone in a separate segment. Then, the yellow saddle goatfish algorithm (YSGA) was used to determine whether the smart contract conditions were optimized. Finally, if the customer is authorized, then the entire transaction process is saved in the blockchain's main ledger and secured by a unique hash. The blockchain application makes our system capable of dealing with large numbers of users in a decentralized manner.

Facts:-

Our system has been examined against several recent well-known assaults/attacks, such as falsification, advanced persistent threat, bribery, spoofing, double spending, chosen text, race, and transaction replay attacks, and has proven to overcome them. In terms of the performance evaluation, we obtained an execution time of approximately 0.0056 nanoseconds, 3.75% complexity, and 1500 KB of memory and disk drive, which is considered low compared to that of state-of-the-art research. Thus, our proposed system is highly acceptable for banking sector applications.

our main contribution

Our contributions are as follows:

The level of security can be increased by using the micro-segmentation principle for the first time with financial systems to isolate each process alone in a separate segment, where if one segment is affected by an assault, then the other segments will remain isolated and safe. To the best of our knowledge, this contribution has not been previously studied.

Two phases of authentication are applied: first, smart contract condition detection, and second, hashing and ID detection in BCT. This procedure will make the authentication process more powerful. There will be no entrance to the system from any assaulter unless he/she is verified in both phases.

The increase in the time consumption and execution time of the specified e-banking process depended on the properties of the YSGA, which is considered one of the fastest search algorithms. To the best of our knowledge, this approach has not been previously applied in e-bank systems.

How does the system work:-

The proposed system's hierarchy. Initially, the proposed system determines the type of CR request based on the decision of the 2PC algorithm and then creates a separate segment for each type of e-banking process after the transaction processes are combined alone in a separate segment. The detection phase of the SC's conditions will start based on the fast detection of the YSGA result.

RESULTS:-

Our proposed module has the following characteristics and advantages.

Fast: Because of the use of modern, easy, and accurate algorithms such as the YSGA and the 2PC, our proposed system guarantees that the e-banking procedure will be performed in real-time.Safe and authentic: The proposed module is controlled by the SC conditions in which both the CR and the SR agree on and do not agree, the e-transaction procedure cannot be performed, and our proposed module can be considered safe and secure.

Additionally, because of the usage of the principle of isolation for each process alone in a separate segment, the danger that threatens a specific segment will remain bounded by that segment, and the other segments will work normally.Integrating: The final layer of our proposed module is the hashing and creating blocks for each procedure. Each block has complete information about the whole transaction process plus the number of previous blocks. This approach provided our proposed module with an advantage in terms of integrity.

Speed: Our proposed system is fast due to the speed of the algorithms used. Additionally, after the full analysis shown above, the time required to complete a full transaction is not more than 0.0056 nanoseconds, which is considered fast compared to other similar methods.

#blockchain

[techpointfundamentals]

SQL Interview Questions and Answers - Part 17:

Q152. What is a Transaction in SQL? Q153. What is TCL in SQL? What are the different TCL commands? Q154. What are ACID properties in SQL? Q155. What is the use of SAVEPOINT in Transaction? Q156. What is the difference between COMMIT and SAVE transactions in SQL? Q157. What is the use of @@TRANCOUNT in SQL? Q158. What is the difference between TRANSACTION and DISTRIBUTED TRANSACTION in SQL? Q159. What is a Two-Phase Commit (2PC) in SQL? Why it is required? Q160. What is the SET XACT_ABORT option in SQL? How does it affect the transaction?

[craigbrownphd-blog-blog]

Apache Ignite Transactions Architecture: 2-Phase Commit Protocol

Apache Ignite supports a range of different Application Programming Interfaces (APIs). In this multi-part article series, we will take a more detailed look at how Apache Ignite manages transactions in its key-value API and some of the mechanisms and protocols it supports. Here are topics we will cover in this series: * Ignite transactions and the two-phase commit protocol. * Concurrency modes and isolation levels. * Transaction handling at the level of Ignite persistence (WAL, checkpointing, and more). * Transaction handling at the level of 3rd party persistence. In this first part, we will begin with a discussion of the two-phase commit (2PC) protocol and then look at how this works with various types of cluster nodes. https://goo.gl/c82bGb #DataIntegration #ML

[globalmediacampaign]

How To Inject an Empty XA Transaction in MySQL

If you are using XA transactions, then you’ve likely run into a few replication issues with the 2PCs (2 Phase Commits). Here is a common error we see in Percona’s Managed Services and a few ways to handle it, including injecting an empty XA transaction. Last_Error: Error 'XAER_NOTA: Unknown XID' on query. Default database: 'punisher'. Query: 'XA COMMIT X'1a',X'a1',1' What Does it Mean? It means that replication has tried to commit an XID (XA transaction ID) that does not exist on the server. We can verify that it does not exist by checking: replica1 [localhost:20002] {msandbox} ((none)) > XA RECOVER CONVERT XID; +----------+--------------+--------------+--------+ | formatID | gtrid_length | bqual_length | data   | +----------+--------------+--------------+--------+ |        1 |            1 |            1 | 0x2BB2 | +----------+--------------+--------------+--------+ 1 row in set (0.00 sec) In this case, there is a prepared XA transaction on the server but it is XID X’2B’,X’B2’,1′ not X’1a’,X’a1’,1′. So indeed, the XID does not exist. How Do We Fix It? A few ways… When using anonymous replication, it can be skipped like any other error: SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1; START SLAVE; When using GTIDs, it can be skipped by the typical injecting an empty GTID: SET @@SESSION.GTID_NEXT= ':'; BEGIN; COMMIT; SET GTID_NEXT='AUTOMATIC'; START SLAVE; Another option is that we can inject an empty XA transaction, much like we do with GTID. Then we can resume replication so it can naturally commit that XID. To prepare an empty XA, first copy the SQL + XID from the error. In this case “XA COMMIT X’1a’,X’a1′,1”. Now transform it into three statements, and run them on the erred replica. XA START X'1a',X'a1',1; XA END X'1a',X'a1',1; XA PREPARE X'1a',X'a1',1; This will have created a prepared XA transaction on the server. We can verify by running: replica1 [localhost:20002] {msandbox} ((none)) > XA RECOVER CONVERT XID; +----------+--------------+--------------+--------+ | formatID | gtrid_length | bqual_length | data   | +----------+--------------+--------------+--------+ |        1 |            1 |            1 | 0x2BB2 | |        1 |            1 |            1 | 0x1AA1 | <--- this is the transaction we just created +----------+--------------+--------------+--------+ 2 rows in set (0.00 sec) So, let’s start replication: replica1 [localhost:20002] {msandbox} ((none)) > START SLAVE; ERROR 1399 (XAE07): XAER_RMFAIL: The command cannot be executed when global transaction is in the  PREPARED state Uh oh, now what? When you prepare an XA transaction on a server, your session cannot execute any other SQL. You must disconnect from MySQL, reconnect, then start replication. replica1 [localhost:20002] {msandbox} ((none)) > exit ...reconnect... replica1 [localhost:20002] {msandbox} ((none)) > START SLAVE; Query OK, 0 rows affected (0.02 sec) Regardless of how you handled the error, it is recommended to run a checksum to validate data consistency. How Does This Happen? 2PCs write to the binlogs in two…phases 🙂 The first phase contains the {XA START/transaction SQL/XA END/XA PREPARE}. Think of all those statements as a single GTID. Once the XA PREPARE command has run, that whole transaction is written to the binary log so these statements will always be written together. Example: # at 903 #200908 20:53:35 server id 100  end_log_pos 1004 CRC32 0xd2f9e5c0       Query   thread_id=4     exec_time=0     error_code=0 SET TIMESTAMP=1599598415/*!*/; XA START X'1a',X'a1',1 /*!*/; # at 1004 #200908 20:53:35 server id 100  end_log_pos 1055 CRC32 0xad24c30d       Table_map: `punisher`.`t1` mapped to number 108 # at 1055 #200908 20:53:35 server id 100  end_log_pos 1100 CRC32 0xf7100e24       Write_rows: table id 108 flags: STMT_END_F BINLOG ' T+9XXxNkAAAAMwAAAB8EAAAAAGwAAAAAAAEACHB1bmlzaGVyAAJ0MQACAxIBAAINwySt T+9XXx5kAAAALQAAAEwEAAAAAGwAAAAAAAEAAgAC//wCAAAAmadRTWMkDhD3 '/*!*/; ### INSERT INTO `punisher`.`t1` ### SET ###   @1=2 ###   @2='2020-09-08 20:53:35' # at 1100 #200908 20:53:44 server id 100  end_log_pos 1191 CRC32 0x314c857d       Query   thread_id=4     exec_time=0     error_code=0 SET TIMESTAMP=1599598424/*!*/; XA END X'1a',X'a1',1 /*!*/; # at 1191 #200908 20:53:44 server id 100  end_log_pos 1229 CRC32 0x829495e8       XA PREPARE X'1a',X'a1',1 XA PREPARE X'1a',X'a1',1 /*!*/; Now we have a prepared XA on the source and replicas (which can hold row locks and block other transactions). The transaction can now be committed or rolled back, this depends on the second phase and this is where the problems come in. The second phase commit/rollback can come seconds later, minutes later, days later, or even never. It all depends on when/if the Transaction Manager issues the command. In this case, it was 4 minutes later: # at 1294 #200908 20:57:37 server id 100  end_log_pos 1388 CRC32 0xe38c4e46       Query   thread_id=4     exec_time=0     error_code=0 SET TIMESTAMP=1599598477/*!*/; XA COMMIT X'1a',X'a1',1 /*!*/; There could be hundreds or thousands of other transactions written to the binary log in between the first and second phases. They could even be written to different binlogs. This explanation is just to show how 2PCs work to understand the separate parts of an XA transaction. Summary Now to try and give some answer to “how does this happen?”…it could be from restoring a backup but MySQL did not get told to prepare some XID on the server. Now replication starts and it is reading events from the source’s binlog. Then it comes across an XA COMMIT but that XID was not prepared so it errors. Ultimately, these issues usually come down to some bug (here is one for example). Do you have XAs blocking other transactions? Check out Planes, Trains, and Automobiles: MySQL XA Transactions. Here is another post on how to troubleshoot XA recovery. Percona Server for MySQL is also working on making XA RECOVER CONVERT XID more helpful! https://www.percona.com/blog/2020/09/15/how-to-inject-an-empty-xa-transaction-in-mysql/

[alliedco]

Enterprise Service Bus integration - Allied Consultants

ESB for system integration implies how a system communicates with rest of the system in an ecosystem; boundaries are flexible. Market analysts says the need for integration varies across the industries, as well as the size of the company under a vertical

Sometimes choosing an ESB solution requires some brainstorming. Here we can help you by identifying questions and then seek for their answers.

Here are few questions you need to ask for before choosing ESB solution.

List of questions:

General Requirements

ESB Solution compatible with which server types?

ESB Solution compatible with which operating systems?

Does ESB solution support abstract messaging layer complexities from developers?

Does ESB Solution provide a fully documented API with examples for various scenarios?

Does ESB component tools above ‘Drag and Drop’ type graphical tooling?

Reliability & Availability

Does ESB solution support transactional two- phase commit?

Does ESB tool have the capabilities to ensure that messages are delivered to end-point exactly once?

Does ESB have an automatic rollback inbuilt functionality that will help ensure there is no avenue for data loss in event of any failure on the solution?

WS protocols that ESB supports?

Integration and Flexibility Requirements

Does ESB solution provide support for messaging models/ Multivendor Interoperability for communication and integration?

Does ESB Solution use the Web Service Framework (WSF) including the proprietary enterprise integration (EAI) middleware models?

Does ESB have a registry to keep track of services and the metadata on a service?

Does ESB can solution compose new services?

Does ESB register new services in a configuration-based fashion?

Does ESB solution provide an online directory of services?

Can ESB solution provide connectivity to the existing enterprise middleware e.g. Back office integration system (BOIS)?

Does ESB have the capability to support different language interfaces for service invocations?

Is ESB compatible with varying Application Server environments via standard protocol and mechanisms such as includes but not limited?

Can ESB solution integrate with 3rd party enterprise monitoring tools

Can ESB solution convert between the protocol native to the messaging platform and other protocols?

Can ESB solution programmatically interact with service externally? These services are the web services published within the ESB

Connectivity /Adaptors Requirements

Does ESB solution provide integrated development environment (IDE)?

Does ESB solution provide development framework/Class libraries that can be leveraged on to build adapter templates?

Can ESB provide support for extending back-end applications by enabling connections to mobile applications running on a variety of heterogeneous devices?

Does ESB provide an adapter(s) for the enterprise resource planning solutions?

Messaging and Mediation Requirements

Does ESB have the ability to support multiple styles of message-based interactions (request/reply, conversational, publish and subscribe, asynchronous messaging and others), including reliable, once-only delivery of messages?

Does ESB have the ability to support synchronous and asynchronous between SOA service consumer modules and service provider modules at run-time?

Does ESB Solution have the ability to apply logic to the routing of messages?

Can ESB provide protocol bridging capability that can support automatic mediation across multiple protocols?

Can ESB provide message transformation capability to process XML data?

Will ESB ensure transactional integrity to enable reliable messaging and event processing?

Does ESB solution have the ability to coordinate a distributed transaction across heterogeneous data resources using a two-phase commit (2PC) protocol or compensating transaction?

Does ESB have message queuing functionalities?

Maintainability Requirements

Will ESB solution allow patching or allow changes to a protocol/service while online (i.e. still being accessed by users), without having to restart the ESB?

Does ESB solution support hot updates of services without restarting the system?

Does ESB solution have the capability to ‘auto-deploy’ one or more components to avoid SLA violation?

Does ESB solution have the SLA monitoring functionality and does?

Does ESB solution provide notifications in the event an SLA has been breached?

Security Requirements

Does ESB solution provide a distinction between services that are accessible over the internet?

Does ESB solution support secure communication using modern encryption techniques?

Does ESB solution support secure communication via custom encryption methods e.g. Secure Socket module (SSM) from NIBSS?

Can ESB Solution support authentication in conjunction with native or bespoke security technology, such as a Lightweight Directory Access Protocol (LDAP/S) directory (this is used to identify an individual based on a username and password)?

Is ESB solution WS-I basic profile complaint?

Can ESB solution support secure connection internally/externally with technology components?

Does ESB solution provide support for document signature(s) to grant permission accordingly?

Does ESB solution have the capability to provide authentication aliases for connections?

Does ESB solution have the ability to support a higher level of security and control FTP, SFTP transfer?

Does ESB solution include security capabilities to manage access to the services and other forms of security?

Is ESB solution PCIDSS compliant?

Database Requirements

Does ESB solution support the different databases?

Does ESB solution have the capacity to support automatic Database discovery?

Does ESB solution have the capacity to support automatic creation of web service WSDL?

Does it support a comprehensive audit trail of all transactions?

Audit, History, and Archiving Requirements

Does system have audit trail functionality?

Does system have an inbuilt storage that should have the capacity to house archived audit trails for a maximum of 10years?

Reporting

Does ESB solution provide customizable reports?

Does ESB solution provider Historical user activity, Real-time user activity, Service Calls, Handled Service Calls and Abandoned Service Calls reports?

Training and Support

Do you have online resources or labs that in-house developer can leverage on(hands-on training) to develop/configure services on ESB?

Do you offer on-site technical training for ESB solution?

Do you offer off-site technical training for ESB solution?

Is there a developer certification path?

Does offer 24*7 support for ESB solution

Does support team have the competence to log in remotely and resolve problems (if required)?

  Jahanzaib khan (JZ)

Senior Integration Consultant

Jahanzaib.khan@alliedc.com

  Original Article: http://www.alliedc.com/enterprise-service-bus-integration/

[some-programming-pearls]

How to handle data inconsistencies across microservices?

Scenario: Imagine a shopping application with multiple microservices handling different aspects such as inventory management, order processing, and payment handling. A customer places an order for a product that is in stock, triggering the deduction of the item from the inventory microservice. However, due to network delays or system failures, the order microservice fails to update the order…

[some-programming-pearls]

Explain what a three-phase commit is in terms of microservice architectures?

In the context of microservices architectures, a Three-Phase Commit (3PC) is a distributed transaction protocol used to coordinate transactions across multiple microservices or distributed components. Similar to the Two-Phase Commit (2PC) protocol, 3PC aims to ensure atomicity and consistency of transactions, but with additional optimizations to address some of the limitations of 2PC. Here’s an…